IPC-Team: Thank you for taking the time to speak with us today, Larry. Can you introduce yourself for our readers?

Larry Garfield: Hi, I’m Larry Garfield, 20+ year PHP veteran. I’ve worked on a couple of different Free Software projects, and currently work as a Staff Engineer for LegalZoom. I am also a leading member of the PHP Framework Interoperability Group (PHP-FIG).

IPC-Team: Congratulations on the recent release of Serde 1.0.0. How did Serde come about and what was your motivation?

Larry Garfield: Serde came out of a need I had while working for TYPO3, the German Free Software CMS. I wanted a tool to help transition TYPO3 from giant global array blobs for all configuration toward well-typed, explicitly defined objects. Translating arrays into objects is basically a serialization problem, so rather than do something one-off I figured it was a good task for serialization.

I first looked at Symfony Serializer, as TYPO3 already uses a number of Symfony components and it was generally regarded as the most robust option. Unfortunately, after spending multiple weeks trying to coax it into doing what I needed I determined that is just couldn’t. It didn’t have the structure-manipulation features I needed, and its architecture was simply too convoluted to make adding it feasible. That meant I had to build my own.

After some initial experimentation of my own, I looked into Rust’s Serde crate, as it’s generally regarded as the best serializer on the market. Rust, of course, is not the same as PHP, but I was still able to draw a lot of ideas from it. For instance, Crell/Serde is streaming, like Rust’s Serde. It doesn’t have a unified in-memory intermediate representation (necessarily), but there is a fixed set of “low level” types that exporters and importers can map to. (That list is smaller for PHP than for Rust, naturally.)

It took a few months, but I was able to get Crell/Serde to do nearly everything I needed it to for TYPO3. Most especially, I’m very happy with the data restructuring capabilities it has. That is, the serialized form of an object doesn’t have to be precisely the same as the in-memory object. There’s robust rules for automatically changing that structure in controlled ways when serializing and deserializing. For instance, a set of 10 JSON properties can be grouped up into three different object properties, with changed names in PHP to avoid prefixes and such, automatically. That was important for TYPO3, because the old array structures had a lot of legacy debt in their design, and this was an opportunity to clean that up.

Along the way, Serde also spawned two supporting libraries: Crell/fp and Crell/AttributeUtils. The latter is where a lot of Serde’s power lives, in fact, in its ability to power nearly everything through PHP attributes. That functionality is now available to any library to use, not just Serde.

In the end, TYPO3 chose not to pursue the array-to-object transition after all. But since it’s an Open Source organization, the code was already written and could be released. After I left TYPO3, I polished the library up a bit further, added a few more features, and released it.

IPC-Team: Serde shares its name with the Serde framework used with Rust, was that an inspiration? Have the two ever been confused?

Larry Garfield: As noted above, yes, it’s definitely named in honor of Rust Serde and drew from its design. So far the name similarity hasn’t been an issue. I did have someone on Mastodon complain that my name choice was going to hurt SEO, but so far that doesn’t seem to have been an issue. It’s too late to change it anyway. 🙂

“Despite all of its power and flexibility, Serde is pretty fast. The last time I benchmarked it, it was faster than Symfony Serializer on the same tasks, despite having more features and options.”

IPC-Team: What formats does Serde support?

Larry Garfield: As of 1.0.0, Crell/Serde can round-trip (serialize and deserialize) PHP arrays, JSON, YAML, and CSV. It can also serialize to JSON and CSV in a streaming-fashion. I have a working branch on XML support, but that’s considerably more challenging and I suspect XML may be better handled in a different approach than a general purpose serializer.

I would like to add support for TOML, and there has been interest in it, but so far we’ve not found any existing TOML 1.0 parsers for PHP, only for the old 0.4 format. If someone made a good 1.0-compatible library, plugging that into Serde should be pretty simple.

YOU LOVE PHP?

Explore the PHP Core Track

Learn more

Serde is entirely modular, so new formats can be added by third parties easily. That said, I’m happy to colocate supportable formats in Serde itself. Ease of setup and use is a key goal for the project.

IPC-Team: What sets Serde apart from the competition?

Larry Garfield: I think Serde excels in a number of areas.

• a. As mentioned, the data-restructuring capabilities are beyond anything else in PHP right now, as far as I’m aware. I think it may even be more flexible than Rust Serde in some ways.
• b. Support for streaming JSON and CSV output. Combined with the ability to read from generators in PHP, that means Serde has effectively no maximum on the size of data it can serialize.
• c. It’s “batteries included.” Symfony Serializer is a bit tricky to setup if you’re using it outside of the Symfony framework itself. There’s lots of exposed moving parts. Serde can be used by just instantiating one class and using it. It can be configured in more robust ways, but for just getting started it’s trivially easy to use.
• d. Despite all of its power and flexibility, Serde is pretty fast. The last time I benchmarked it, it was faster than Symfony Serializer on the same tasks, despite having more features and options. If you don’t need Serde’s capabilities than a purpose-build lightweight hydrator would still be faster, but in most cases Serde will be fast enough to just use and move on. It also has natural places to hook in and provide custom serialization for certain objects, which can be purpose-built and faster than the general pipeline.
• e. “Scopes” support. Symfony Serializer also supports multiple ways of serializing an object through serialization groups, which are very similar. The way Serde ties in attributes, however, gives it even more flexibility, and I am not aware of any other serializer besides Serde and Symfony that have that ability.
• f. This is more of a personal victory, but Serde is about 99% functionally pure. It follows the functional principles of immutable variables, functionally pure methods, statelessness, etc., even though it’s overall object-oriented. Really holding to that line helped drive the architecture in a very good place, and is one of the reasons Serde is so extensible.

IPC-Team: What are some of the advantages of using Serde for serialization and deserialization in PHP applications?

Larry Garfield: I see Crell/Serde as a good fit any time unstructured data is coming into an application. It is always better to be working with well-structured, well-typed objects than array blobs. Because Serde is so robust and fast, it’s straightforward to “guard” everywhere data is coming into the application (from an HTTP request, config file, database, REST response from another service, etc.) with a deserialization layer that ensures you have well-structured, typed, autocompletable data to work with. That can drastically cut down on the amount of error handling needed elsewhere the application.

The same is true when sending requests. Rather than manually build up an array to pass to some API call (as many API bridges expect you to do), you can build up a well-structured object, using all of the good OOP techniques you already know (typed properties, methods, etc.), and then dump that to JSON or a PHP array at the last second before sending it over the wire. That ensures you have the right structured data every time; your PHP code wouldn’t even run otherwise.

IPC-Team: What are some of the Serde’s limitations?

Larry Garfield: As mentioned, XML and TOML support are still pending. I’ve had someone ask about binary formats like protobuf, and I think that could probably be done, but I’ve not tried.

There are some edge cases some users have reported around the data restructuring logic when using “boxed” value objects. For instance, a “Name” class that contains just a string and an “Email” class that contains just a string, both of which are then properties on an object to serialize. That’s only partially supported right now, although I’m working on ways to improve it. Hopefully it will be resolved by the time you read this.

Crell/Serde also supports only objects. It cannot serialize directly from or to arrays or primitives. In practice I don’t think that’s a big issue, as “turning unstructured data into structured objects” is the entire reason it exists.

IPC-Team: How do you recommend getting started with Serde and how can someone get involved with the community?

Larry Garfield: I’m quite proud of the Serde documentation in the project README. It’s long, but very complete and detailed and gradually works up to more and more features. The best way to get started is to read the first section or two, then try playing with it. It’s deliberately easy to just toy around with, and add-in capabilities as you find a use for them.

As far as getting involved in the project itself, as in any Free Software project, file good bug reports, file good feature requests. If you want to try and add a feature, please open an issue first to discuss it. I don’t want someone wasting time on a feature or design that won’t work.

In particular, if someone wants to try writing a formatter for writing to protobuf or other binary formats, I’d love to see what can be done there. I’ve not worked with that format myself so that’s a good place to dig in.

At the moment, Crell/Serde is entirely volunteer-developed by me, since it’s no longer sponsored by TYPO3. Please keep that in mind any time you’re working with this or any Free Software project. Of course, if you are interested, I’m happy to accept sponsorship for prioritizing certain requests.

IPC-Team: What’s on your wishlist for future iterations or updates of Serde?

Larry Garfield: Mainly addressing the limitations mentioned above. TOML support would be good to include. XML may or may not make sense. I like the idea of supporting boxed value objects better. Binary formats would be another good differentiating feature.

One feature in particular I’m exploring is allowing attributes to be read from a non-attribute source. AttributeUtils, which handles all attribute parsing, is also clean enough that plugging in an alternate backend should be easy. If that alternate backend reads data from a YAML file, for instance, using Serde, and deserializes into whatever attribute set a given library is using (such as Serde’s own attributes), that would allow any AttributeUtils-using library to easily support YAML or JSON configuration in addition to in-code attributes, but still resulting in the same metadata objects for a library to use. I’m still working on how to make this work, but I’m pretty sure it is feasible. Stay tuned.